{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Binary Quantization with Qdrant & OpenAI Embedding\n",
    "\n",
    "---\n",
    "In the world of large-scale data retrieval and processing, efficiency is crucial. With the exponential growth of data, the ability to retrieve information quickly and accurately can significantly affect system performance. This blog post explores a technique known as binary quantization applied to OpenAI embeddings, demonstrating how it can enhance **retrieval latency by 20x** or more.\n",
    "\n",
    "## What Are OpenAI Embeddings?\n",
    "OpenAI embeddings are numerical representations of textual information. They transform text into a vector space where semantically similar texts are mapped close together. This mathematical representation enables computers to understand and process human language more effectively.\n",
    "\n",
    "## Binary Quantization\n",
    "Binary quantization is a method which converts continuous numerical values into binary values (0 or 1). It simplifies the data structure, allowing faster computations. Here's a brief overview of the binary quantization process applied to OpenAI embeddings:\n",
    "\n",
    "1. **Load Embeddings**: OpenAI embeddings are loaded from parquet files.\n",
    "2. **Binary Transformation**: The continuous valued vectors are converted into binary form. Here, values greater than 0 are set to 1, and others remain 0.\n",
    "3. **Comparison & Retrieval**: Binary vectors are used for comparison using logical XOR operations and other efficient algorithms.\n",
    "\n",
    "Binary Quantization is a promising approach to improve retrieval speeds and reduce memory footprint of vector search engines. In this notebook we will show how to use Qdrant to perform binary quantization of vectors and perform fast similarity search on the resulting index.\n",
    "\n",
    "## Table of Contents\n",
    "1. Imports\n",
    "2. Download and Slice Dataset\n",
    "3. Create Qdrant Collection\n",
    "4. Indexing\n",
    "5. Search\n",
    "\n",
    "## 1. Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-06-06T18:31:54.357040Z",
     "start_time": "2024-06-06T18:31:52.672431Z"
    }
   },
   "outputs": [],
   "source": [
    "!pip install qdrant-client pandas dataset --quiet --upgrade"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-06-06T18:31:55.482034Z",
     "start_time": "2024-06-06T18:31:54.357645Z"
    }
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import random\n",
    "import time\n",
    "\n",
    "import datasets\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from qdrant_client import QdrantClient, models\n",
    "\n",
    "random.seed(37)\n",
    "np.random.seed(37)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Download and Slice Dataset\n",
    "\n",
    "We will be using the [dbpedia-entities](https://huggingface.co/datasets/Qdrant/dbpedia-entities-openai3-text-embedding-3-small-1536-100K) dataset from the [HuggingFace Datasets](https://huggingface.co/datasets) library. This contains 100K vectors of 1536 dimensions each"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-06-06T18:31:58.405581Z",
     "start_time": "2024-06-06T18:31:55.471027Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "100000"
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset = datasets.load_dataset(\n",
    "    \"Qdrant/dbpedia-entities-openai3-text-embedding-3-small-1536-100K\", split=\"train\"\n",
    ")\n",
    "len(dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-06-06T18:32:24.456211Z",
     "start_time": "2024-06-06T18:31:58.396924Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "1536"
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "n_dim = len(dataset[\"text-embedding-3-small-1536-embedding\"][0])\n",
    "n_dim"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-06-06T18:32:24.536891Z",
     "start_time": "2024-06-06T18:32:24.455087Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "True"
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "client = QdrantClient(  # assumes Qdrant is launched at localhost:6333\n",
    "    prefer_grpc=True,\n",
    ")\n",
    "\n",
    "collection_name = \"binary-quantization\"\n",
    "\n",
    "client.create_collection(\n",
    "    collection_name=collection_name,\n",
    "    vectors_config=models.VectorParams(\n",
    "        size=n_dim,\n",
    "        distance=models.Distance.DOT,\n",
    "        on_disk=True,\n",
    "    ),\n",
    "    quantization_config=models.BinaryQuantization(\n",
    "        binary=models.BinaryQuantizationConfig(always_ram=True),\n",
    "    ),\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-06-06T18:33:22.583349Z",
     "start_time": "2024-06-06T18:32:24.815155Z"
    }
   },
   "outputs": [],
   "source": [
    "def iter_dataset(dataset):\n",
    "    for point in dataset:\n",
    "        yield point[\"openai\"], {\"text\": point[\"text\"]}\n",
    "\n",
    "\n",
    "vectors, payload = zip(*iter_dataset(dataset))\n",
    "client.upload_collection(\n",
    "    collection_name=collection_name,\n",
    "    vectors=vectors,\n",
    "    payload=payload,\n",
    "    parallel=max(1, (os.cpu_count() // 2)),\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-06-06T18:35:47.748898Z",
     "start_time": "2024-06-06T18:35:47.740719Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "{'status': <CollectionStatus.GREEN: 'green'>,\n 'optimizer_status': <OptimizersStatusOneOf.OK: 'ok'>,\n 'vectors_count': None,\n 'indexed_vectors_count': 97760,\n 'points_count': 100000,\n 'segments_count': 7,\n 'config': {'params': {'vectors': {'size': 1536,\n    'distance': <Distance.DOT: 'Dot'>,\n    'hnsw_config': None,\n    'quantization_config': None,\n    'on_disk': True,\n    'datatype': None},\n   'shard_number': 1,\n   'sharding_method': None,\n   'replication_factor': 1,\n   'write_consistency_factor': 1,\n   'read_fan_out_factor': None,\n   'on_disk_payload': True,\n   'sparse_vectors': None},\n  'hnsw_config': {'m': 16,\n   'ef_construct': 100,\n   'full_scan_threshold': 10000,\n   'max_indexing_threads': 0,\n   'on_disk': False,\n   'payload_m': None},\n  'optimizer_config': {'deleted_threshold': 0.2,\n   'vacuum_min_vector_number': 1000,\n   'default_segment_number': 0,\n   'max_segment_size': None,\n   'memmap_threshold': None,\n   'indexing_threshold': 20000,\n   'flush_interval_sec': 5,\n   'max_optimization_threads': None},\n  'wal_config': {'wal_capacity_mb': 32, 'wal_segments_ahead': 0},\n  'quantization_config': {'binary': {'always_ram': True}}},\n 'payload_schema': {}}"
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "collection_info = client.get_collection(collection_name=f\"{collection_name}\")\n",
    "collection_info.dict()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Oversampling vs Recall\n",
    "\n",
    "### Preparing a query dataset\n",
    "\n",
    "For the purpose of this illustration, we'll take a few vectors which we know are already in the index and query them. We should get the same vectors back as results from the Qdrant index. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-06-06T18:33:22.643845Z",
     "start_time": "2024-06-06T18:33:22.589346Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "[89391,\n 79659,\n 12006,\n 80978,\n 87219,\n 97885,\n 83155,\n 67504,\n 4645,\n 82711,\n 48395,\n 57375,\n 69208,\n 14136,\n 89515,\n 59880,\n 78730,\n 36952,\n 49620,\n 96486,\n 55473,\n 58179,\n 18926,\n 6489,\n 11931,\n 54146,\n 9850,\n 71259,\n 37825,\n 47331,\n 84964,\n 92399,\n 56669,\n 77042,\n 73744,\n 47993,\n 83780,\n 92429,\n 75114,\n 4463,\n 69030,\n 81185,\n 27950,\n 66217,\n 54652,\n 8260,\n 1151,\n 993,\n 85954,\n 66863,\n 47303,\n 8992,\n 92688,\n 76030,\n 29472,\n 3077,\n 42454,\n 46120,\n 69140,\n 20877,\n 2844,\n 95423,\n 1770,\n 28568,\n 96448,\n 94227,\n 40837,\n 91684,\n 29785,\n 66936,\n 85121,\n 39546,\n 81910,\n 5514,\n 37068,\n 35731,\n 93990,\n 26685,\n 63076,\n 18762,\n 27922,\n 34916,\n 80976,\n 83189,\n 6328,\n 57508,\n 58860,\n 13758,\n 72976,\n 85030,\n 332,\n 34963,\n 85009,\n 31344,\n 11560,\n 58108,\n 85163,\n 17064,\n 44712,\n 45962]"
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "query_indices = random.sample(range(len(dataset)), 100)\n",
    "query_dataset = dataset[query_indices]\n",
    "query_indices"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-06-06T18:33:22.644389Z",
     "start_time": "2024-06-06T18:33:22.642909Z"
    }
   },
   "outputs": [],
   "source": [
    "## Add Gaussian noise to any vector\n",
    "\n",
    "\n",
    "def add_noise(vector, noise=0.05):\n",
    "    return vector + noise * np.random.randn(*vector.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-06-06T18:33:22.647634Z",
     "start_time": "2024-06-06T18:33:22.646107Z"
    }
   },
   "outputs": [],
   "source": [
    "def correct(results, text):\n",
    "    return text in [x.payload[\"text\"] for x in results]\n",
    "\n",
    "\n",
    "def count_correct(query_dataset, limit=1, oversampling=1, rescore=False):\n",
    "    correct_results = 0\n",
    "    for query_vector, text in zip(query_dataset[\"openai\"], query_dataset[\"text\"]):\n",
    "        results = client.search(\n",
    "            collection_name=collection_name,\n",
    "            query_vector=add_noise(np.array(query_vector)),\n",
    "            limit=limit,\n",
    "            search_params=models.SearchParams(\n",
    "                quantization=models.QuantizationSearchParams(\n",
    "                    rescore=rescore,\n",
    "                    oversampling=oversampling,\n",
    "                )\n",
    "            ),\n",
    "        )\n",
    "        correct_results += correct(results, text)\n",
    "    return correct_results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-06-06T18:36:09.037565Z",
     "start_time": "2024-06-06T18:35:58.069484Z"
    },
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "limit_grid = [1, 3, 10, 20, 50]\n",
    "oversampling_grid = [1.0, 3.0, 5.0]\n",
    "rescore_grid = [True, False]\n",
    "results = []\n",
    "\n",
    "for limit in limit_grid:\n",
    "    for oversampling in oversampling_grid:\n",
    "        for rescore in rescore_grid:\n",
    "            start = time.perf_counter()\n",
    "            correct_results = count_correct(\n",
    "                query_dataset, limit=limit, oversampling=oversampling, rescore=rescore\n",
    "            )\n",
    "            end = time.perf_counter()\n",
    "            results.append(\n",
    "                {\n",
    "                    \"limit\": limit,\n",
    "                    \"oversampling\": oversampling,\n",
    "                    \"bq_candidates\": int(oversampling * limit),\n",
    "                    \"rescore\": rescore,\n",
    "                    \"accuracy\": correct_results / 100,\n",
    "                    \"total queries\": len(query_dataset[\"text\"]),\n",
    "                    \"time\": end - start,\n",
    "                }\n",
    "            )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-06-06T18:36:09.054593Z",
     "start_time": "2024-06-06T18:36:09.048053Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>limit</th>\n      <th>oversampling</th>\n      <th>rescore</th>\n      <th>accuracy</th>\n      <th>bq_candidates</th>\n      <th>time</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>1.0</td>\n      <td>True</td>\n      <td>0.95</td>\n      <td>1</td>\n      <td>0.300152</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1</td>\n      <td>1.0</td>\n      <td>False</td>\n      <td>0.85</td>\n      <td>1</td>\n      <td>0.244668</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1</td>\n      <td>3.0</td>\n      <td>True</td>\n      <td>0.95</td>\n      <td>3</td>\n      <td>0.124406</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1</td>\n      <td>3.0</td>\n      <td>False</td>\n      <td>0.83</td>\n      <td>3</td>\n      <td>0.171471</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1</td>\n      <td>5.0</td>\n      <td>True</td>\n      <td>0.98</td>\n      <td>5</td>\n      <td>0.118219</td>\n    </tr>\n    <tr>\n      <th>5</th>\n      <td>1</td>\n      <td>5.0</td>\n      <td>False</td>\n      <td>0.87</td>\n      <td>5</td>\n      <td>0.111914</td>\n    </tr>\n    <tr>\n      <th>6</th>\n      <td>3</td>\n      <td>1.0</td>\n      <td>True</td>\n      <td>0.95</td>\n      <td>3</td>\n      <td>0.121328</td>\n    </tr>\n    <tr>\n      <th>7</th>\n      <td>3</td>\n      <td>1.0</td>\n      <td>False</td>\n      <td>0.92</td>\n      <td>3</td>\n      <td>0.267725</td>\n    </tr>\n    <tr>\n      <th>8</th>\n      <td>3</td>\n      <td>3.0</td>\n      <td>True</td>\n      <td>0.96</td>\n      <td>9</td>\n      <td>0.416834</td>\n    </tr>\n    <tr>\n      <th>9</th>\n      <td>3</td>\n      <td>3.0</td>\n      <td>False</td>\n      <td>0.90</td>\n      <td>9</td>\n      <td>0.410730</td>\n    </tr>\n    <tr>\n      <th>10</th>\n      <td>3</td>\n      <td>5.0</td>\n      <td>True</td>\n      <td>0.97</td>\n      <td>15</td>\n      <td>0.231671</td>\n    </tr>\n    <tr>\n      <th>11</th>\n      <td>3</td>\n      <td>5.0</td>\n      <td>False</td>\n      <td>0.93</td>\n      <td>15</td>\n      <td>0.252269</td>\n    </tr>\n    <tr>\n      <th>12</th>\n      <td>10</td>\n      <td>1.0</td>\n      <td>True</td>\n      <td>0.96</td>\n      <td>10</td>\n      <td>0.133462</td>\n    </tr>\n    <tr>\n      <th>13</th>\n      <td>10</td>\n      <td>1.0</td>\n      <td>False</td>\n      <td>0.92</td>\n      <td>10</td>\n      <td>0.285158</td>\n    </tr>\n    <tr>\n      <th>14</th>\n      <td>10</td>\n      <td>3.0</td>\n      <td>True</td>\n      <td>0.95</td>\n      <td>30</td>\n      <td>0.320695</td>\n    </tr>\n    <tr>\n      <th>15</th>\n      <td>10</td>\n      <td>3.0</td>\n      <td>False</td>\n      <td>0.98</td>\n      <td>30</td>\n      <td>0.457904</td>\n    </tr>\n    <tr>\n      <th>16</th>\n      <td>10</td>\n      <td>5.0</td>\n      <td>True</td>\n      <td>0.96</td>\n      <td>50</td>\n      <td>0.453204</td>\n    </tr>\n    <tr>\n      <th>17</th>\n      <td>10</td>\n      <td>5.0</td>\n      <td>False</td>\n      <td>0.94</td>\n      <td>50</td>\n      <td>0.450944</td>\n    </tr>\n    <tr>\n      <th>18</th>\n      <td>20</td>\n      <td>1.0</td>\n      <td>True</td>\n      <td>0.97</td>\n      <td>20</td>\n      <td>0.361066</td>\n    </tr>\n    <tr>\n      <th>19</th>\n      <td>20</td>\n      <td>1.0</td>\n      <td>False</td>\n      <td>0.95</td>\n      <td>20</td>\n      <td>0.585992</td>\n    </tr>\n    <tr>\n      <th>20</th>\n      <td>20</td>\n      <td>3.0</td>\n      <td>True</td>\n      <td>0.96</td>\n      <td>60</td>\n      <td>0.550389</td>\n    </tr>\n    <tr>\n      <th>21</th>\n      <td>20</td>\n      <td>3.0</td>\n      <td>False</td>\n      <td>0.96</td>\n      <td>60</td>\n      <td>0.618630</td>\n    </tr>\n    <tr>\n      <th>22</th>\n      <td>20</td>\n      <td>5.0</td>\n      <td>True</td>\n      <td>1.00</td>\n      <td>100</td>\n      <td>0.458241</td>\n    </tr>\n    <tr>\n      <th>23</th>\n      <td>20</td>\n      <td>5.0</td>\n      <td>False</td>\n      <td>0.95</td>\n      <td>100</td>\n      <td>0.441106</td>\n    </tr>\n    <tr>\n      <th>24</th>\n      <td>50</td>\n      <td>1.0</td>\n      <td>True</td>\n      <td>0.98</td>\n      <td>50</td>\n      <td>0.603967</td>\n    </tr>\n    <tr>\n      <th>25</th>\n      <td>50</td>\n      <td>1.0</td>\n      <td>False</td>\n      <td>0.96</td>\n      <td>50</td>\n      <td>0.514531</td>\n    </tr>\n    <tr>\n      <th>26</th>\n      <td>50</td>\n      <td>3.0</td>\n      <td>True</td>\n      <td>1.00</td>\n      <td>150</td>\n      <td>0.548153</td>\n    </tr>\n    <tr>\n      <th>27</th>\n      <td>50</td>\n      <td>3.0</td>\n      <td>False</td>\n      <td>0.98</td>\n      <td>150</td>\n      <td>0.608930</td>\n    </tr>\n    <tr>\n      <th>28</th>\n      <td>50</td>\n      <td>5.0</td>\n      <td>True</td>\n      <td>1.00</td>\n      <td>250</td>\n      <td>0.487522</td>\n    </tr>\n    <tr>\n      <th>29</th>\n      <td>50</td>\n      <td>5.0</td>\n      <td>False</td>\n      <td>0.99</td>\n      <td>250</td>\n      <td>0.313810</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
      "text/plain": "    limit  oversampling  rescore  accuracy  bq_candidates      time\n0       1           1.0     True      0.95              1  0.300152\n1       1           1.0    False      0.85              1  0.244668\n2       1           3.0     True      0.95              3  0.124406\n3       1           3.0    False      0.83              3  0.171471\n4       1           5.0     True      0.98              5  0.118219\n5       1           5.0    False      0.87              5  0.111914\n6       3           1.0     True      0.95              3  0.121328\n7       3           1.0    False      0.92              3  0.267725\n8       3           3.0     True      0.96              9  0.416834\n9       3           3.0    False      0.90              9  0.410730\n10      3           5.0     True      0.97             15  0.231671\n11      3           5.0    False      0.93             15  0.252269\n12     10           1.0     True      0.96             10  0.133462\n13     10           1.0    False      0.92             10  0.285158\n14     10           3.0     True      0.95             30  0.320695\n15     10           3.0    False      0.98             30  0.457904\n16     10           5.0     True      0.96             50  0.453204\n17     10           5.0    False      0.94             50  0.450944\n18     20           1.0     True      0.97             20  0.361066\n19     20           1.0    False      0.95             20  0.585992\n20     20           3.0     True      0.96             60  0.550389\n21     20           3.0    False      0.96             60  0.618630\n22     20           5.0     True      1.00            100  0.458241\n23     20           5.0    False      0.95            100  0.441106\n24     50           1.0     True      0.98             50  0.603967\n25     50           1.0    False      0.96             50  0.514531\n26     50           3.0     True      1.00            150  0.548153\n27     50           3.0    False      0.98            150  0.608930\n28     50           5.0     True      1.00            250  0.487522\n29     50           5.0    False      0.99            250  0.313810"
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(results)\n",
    "\n",
    "df[[\"limit\", \"oversampling\", \"rescore\", \"accuracy\", \"bq_candidates\", \"time\"]]\n",
    "# df.to_csv(\"candidates-rescore-time.csv\", index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### Why results for oversampling=1.0 and limit=1 with rescore=True are better than with rescore=False? \n",
    "\n",
    "It might seem that with oversampling=1.0 and limit=1 Qdrant retrieves only 1 point, and it does not matter whether we rescore it or not, it should stay the same, but with a different score (from original vectors).\n",
    "\n",
    "But in fact, there are 2 reasons why results are different:\n",
    "1) HNSW is an approximate algorithm, and it might return different results for the same query. \n",
    "2) Qdrant stores points in segments. When we do a query for 1 point, Qdrant looks for this one point in each segment, and then chooses the best match. \n",
    "3) In this example we had 8 segments, Qdrant found 8 points with binary scores, replaced their scores with original vectors scores, and selected the best one from them, which led to a better accuracy. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
